Skip to content

REQ: limit N and keep it consistent after a failed merge; KLL items merge min/max ordering - #809

Open
leerho wants to merge 1 commit into
mainfrom
req-n-limit-merge-safety
Open

leerho wants to merge 1 commit into
mainfrom
req-n-limit-merge-safety

Conversation

@leerho

@leerho leerho commented Nov 20, 2026

Copy link
Copy Markdown
Member

Java counterparts of issues found while reviewing ds-cpp #587. Independent of #808 (no conflict): #808 covers update() and compress(), this PR covers the N limit and merge().

REQ

No limit on N. update() did totalN++ and merge() did totalN += other.totalN with no check, so repeated merges could wrap N negative. KLL already uses Math.addExact; C++ now caps N at 2^63-1.

  • merge() throws SketchesArgumentException if the combined N would exceed Long.MAX_VALUE. The sketch is unchanged.
  • update() throws SketchesStateException if N is already Long.MAX_VALUE. The sketch is unchanged. NaN updates are still ignored.

A failed merge left N out of step. merge() added the other sketch's N and set min and max before merging the compactors. If that failed partway, N counted items that were never merged.

  • N, min and max are now set after the compactors merge.
  • If the compactor merge throws, N is advanced by the weight actually merged (each item at level h weighs 2^h), and min and max include the other sketch's only if some of its items were merged. This matches C++ Mostly spelling errors and documentation corrections. #587.

Both methods' javadoc now names the exceptions.

KLL

I checked the four KLL merge implementations for failures partway through.

  • Doubles, floats, longs: no change needed. Level-0 items go in through single-item updates, which are safe since KLL: keep the sketch consistent when a single-item update fails #806. The higher levels are compacted into new arrays, and nothing is committed until the allocations, including any MemorySegmentRequest, have succeeded.
  • Items: the new min and max were computed with the comparator after N, the levels and the items were committed. A throwing comparator there left the merged items outside the min and max. The comparisons now happen before anything is committed.

Testing

  • ReqMergeLimitsTest:
    • mergeRejectsNAboveMax and updateRejectsNAboveMax: the sketch image is unchanged after the exception, and a merge can reach exactly Long.MAX_VALUE.
    • partialMergeKeepsNConsistent: the other sketch's third compactor has the wrong lgWeight, so the merge fails after two levels. N must equal the original N plus exactly the merged weight, min must include the other's, the iterator weight must equal N, and the sketch keeps working and round-trips.
  • KllMergeFailureTest: a dry run counts the merge's comparisons. Then, on identical seeded sketches, the comparator throws at the min comparison. Every retained item must lie within min and max, and the iterator weight must equal N.
  • All four tests fail on main. On main the KLL one leaves item 000442 below min 000990.
  • Full suite: 2,452 tests, 0 failures. javac -Xlint:all -Xdoclint:all,-missing reports nothing in the changed files.

🤖 Generated with Claude Code

…erge; KLL items merge compares min and max before committing

REQ
- merge() throws SketchesArgumentException, with the sketch unchanged,
  if the combined N would exceed Long.MAX_VALUE.
- update() throws SketchesStateException, with the sketch unchanged,
  if N is already Long.MAX_VALUE.
- merge() sets N, min and max after the compactors are merged. If that
  merge is interrupted, N is advanced by the weight actually merged, and
  min and max include the other sketch's only if items were merged.
- Javadoc for both methods names the exceptions.

KLL
- The items merge computes the new min and max before it commits N, the
  levels and the items, so a comparator that throws there leaves the
  sketch as it was after the level-0 updates. The doubles, floats and
  longs merges were checked and need no change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant